Can constant treatment effects within subgroup explain heterogeneity in welfare reform effects?” Working paper

The welfare reform literature of the last decade has largely focused on mean treatment effects. Our own recent work, Bitler, Gelbach & Hoynes (2006), estimates quantile treatment effects (QTE) and provides evidence of pervasive heterogeneity in the effects of Connecticut’s Jobs First. To the extent that other research in the welfare reform literature has addressed treatment effect heterogeneity, the focus has largely been on mean treatment effects across demographic subgroups. In this paper, we again use experimental data from Jobs First and comprehensively evaluate whether subgroup-specific mean treatment effects can replicate the treatment effect heterogeneity we measured previously. We begin by presenting mean treatment effects by subgroup and testing for differences. We then present estimates of QTE within subgroup, to explore the within group heterogeneity. Finally, we compare actual QTE to “synthetic ” QTE that we construct under the assumption of constant mean impacts. With this, we develop and implement a formal test of the sufficiency of constant mean impacts. We have three major findings. First, mean impacts fail to replicate the strong predictions of labor supply theory that are evident in the QTE. Second, the adequacy of the subgroup mean impacts in replicating the QTE is improved substantially when we account for differences in extensive margin impacts of reform across groups. Finally, we find that mean treatment effects for subgroups based on welfare participants ’ earnings history come closest to matching the labor supply predictions while the mean effects for subgroups based on standard demographic variables (race, education) are much less successful